1. Field
This disclosure relates to digital displays, more particularly to methods for extending the bit depth of displays exploiting aspects of the human visual system.
2. Background
Continuous tone, or contone, imagery usually has 24 bits/pixel as a minimum. Eight bits are allocated for each color in typical displays. However, lower cost displays have limitations on the number of bits they can have per pixel. The limitations come from limited memory in video random access memory (VRAM), the characteristics of the display itself, or from digital to analog converters (DAC) used in some cathode ray tube (CRT) displays.
For example, in many laptop computers the highest gray-level resolution is often the ‘thousands of colors’ mode. This mode corresponds to 16 bits/pixel, rather than the 24 bits/pixel in larger computers or those having more VRAM. The 16 bits are typically allocated 5 bits for red, 6 bits for green and 5 bits for blue. In lesser quality products only 15 bits/pixel are used, with 5 bits allocated per color. The number of bits per pixel per color will be referred to as bit-depth.
In displays having limited bit-depth, contour artifacts appear in smooth gradient regions. For example, an image that includes part of the sky will show visible contour lines in the smooth gradient blue regions of the sky. Previous and current techniques for reduction of these artifacts exist.
L. G. Roberts performed some of the original work in this area in the area of contour prevention for pulse code modulation (PCM) coded and transmitted images. As this was the beginning of image compression efforts, it was a significant achievement to compress the image from 7 bits/pixel to 2 or 3 bits/pixel. The compression technique used a gray level resolution reduction via amplitude quantization. The primary distortion was the contouring artifacts, often manifested as false edges in slowly varying gradients.
In the Roberts technique, show in prior art FIG. 1, a one-dimensional white noise sequence that is predetermined is added to an image during a raster scan prior to quantization. Because the noise sequence is predetermined it is often referred to as pseudo-random noise. To avoid detracting from the image quality, the noise is removed after it is received, just prior to display of the data to which it was added. The subtracted noise is in phase with and identical to the transmitter noise. The noise effectively breaks up the contouring artifacts.
At the time this technique was promulgated, the breaking up of contouring artifacts was an empirical observation. However, it has come to be understood that the elements along the contour are varied in their orientation by the noise, so that they fall outside the association field for the visual system. The quantization process leaves some noise in the image, since it occurs between the steps of adding and then subtracting the noise.
Roberts' work has been largely forgotten as a compression technique. The newer techniques of digital pulse code modulation (DPCM), discrete cosine transform (DCT) and wavelet compression have enabled larger amounts of compression without contouring artifacts. The DCT and wavelet techniques accomplish this primarily by shifting the compression process from the spatial domain to the frequency domain.
Application of the Roberts' method can be found in U.S. Pat. Nos. 3,244,808, 3,562,420 and 3,739,082. The first patent, issued in 1966, implements a system similar to that shown in prior art FIG. 1. In this patent, the distribution of the noise is uniform and assumed to be white noise. The second patent builds on the first and includes an embodiment in which the noise is high-pass along both the spatial and temporal dimensions. In the third patent listed above, the system adds noise as in Roberts' approach, but does not remove noise at the receiver. The applied noise is in an ordered pattern.
These techniques are typically referred to as microdither to differentiate it from dither, a term more commonly applied to halftoning techniques. Halftoning dither is a spatial dither, but the microdither is an amplitude dither. A great deal of work has been done in halftoning, both for displays and printing applications. The references fall generally in two categories, either general dithering approaches using noise, or approaches specifically directed to eliminating the contour artifacts.
A general dithering approach for display systems can be found in U.S. Pat. No. 4,275,411, issued Jun. 23, 1981, and U.S. Pat. No. 3,961,134, issued Jun. 1, 1976. In the '411 patent spatiotemporal dither is used with two-dimensional arrays and included a process to make variations for other frames. In the '134 patent a quantized image is compared against a dither matrix. The dither matrix contains all values of grayscale appearing once, and therefore the size is dependent on the desired grayscale resolution. Other examples suffer from this limitation, as seen by U.S. Pat. No. 5,164,717, issued Nov. 17, 1992.
Other dithering approaches do not have a predetermined size for the dither array. In U.S. Pat. No. 4,758,893, issued Jun. 19, 1988, the size of the dither array is triggered by phase. Additionally, the description refers to characteristics of the human visual system. However, the reference is very general and essentially means that the spatial and temporal frequencies in the dithering patterns are high, as in one embodiment of U.S. Pat. No. 3,562,420, previously mentioned. Similarly, an approach using a dither bitmap to enable an image with a higher number of gray levels to be displayed on an output device with a lower number of gray levels is shown in U.S. Pat. No. 5,333,260, issued in 1994.
Use of the human visual system characteristics can also be found in U.S. Pat. No. 5,619,230, issued Apr. 8, 1997. The noise used is high-pass noise, but is applied in direct proportion to the visual system's sensitivity across frequency. Whether the sensitivity decreases or increases with increasing frequency depends upon the viewing distance and pixel resolution. For most viewing distances and pixel resolutions, the digital Nyquist frequency maps to a cycle/degree higher than 3-5 cycles/degree, which is the typical peak frequency of visual MTF. In this case, sensitivity will decrease with increasing frequency. However for low resolution and close distances, the opposite it true.
Other approaches use high-pass noise, or approximation of it. For example, U.S. Pat. No. 5,111,310, issued May 5, 1992, suggests designing the dithering array in such a manner that the resulting halftone patterns approximate blue (high-pass) noise.
The more common definition of dithering can be found in U.S. Pat. No. 4,956,638, issued Sep. 11, 1990. In this patent, dithering is defined as using a pattern of two colors or gray levels that are close to the desired color or level. When the eye averages the two colors or levels, they appear to be the desired color. This use of more than one bit for dithering is referred to as multi-bit dithering.
Other approaches to multi-bit dithering limit the size of the dither array for each pixel. For example, U.S. Pat. No. 5,138,303, issued Aug. 11, 1992, uses a dither array for each pixel that is 2×2. Other techniques base the size of the dither array on the number of desired gray levels. This is shown in U.S. Pat. No. 5,696,602, issued Dec. 9, 1997, where the dither array size of 16×16 results in 256 levels.
Dithering is applied in general cases as discussed above, and for specific problems. In one case where clipping and gray scale error function were the problems, multi-bit dithering was applied. This is found in U.S. Pat. No. 5,201,030, issued Apr. 6, 1993.
As mentioned above, the specific artifact under discussion is that of contouring, false edges in slowly varying gradients. Several techniques, including the addition of noise as in dithering, have been used to overcome this problem.
One approach is to segment the image into two regions, one of edges and one of non-edges, as shown by U.S. Pat. No. 5,218,649, issued Jun. 8, 1993. Each segment is filtered differently. The intention was for a post-processing for compressed and decompressed images.
Other techniques are directed to a particular compression or decompression technique. For example, U.S. Pat. No. 5,651,078, issued Jul. 22, 1997, addresses contouring artifacts in MPEG (Moving Pictures Experts Group) and MPEG2 schemes. In these schemes, contouring can occur in the dark areas of images. The technique applies different gains to the areas, boosting the captured image noise to break up the artifacts.
Contouring as a particular example of amplitude quantization artifact is discussed in U.S. Pat. No. 5,809,178, issued Sep. 15, 1998. The technique suggests performing an experiment to determine the quantization interval based upon noise already present in the image. It suggests that the interval of noise/quantization should be ⅜.
Finally, contouring artifacts in print are addressed by U.S. Pat. No. 5,920,653, issued Jul. 6, 1999. It uses two channels, one layer is a channel making large dots and the other is a channel making small dots.
However, none of these approaches solves the problem in a robust, computationally simple manner. In addition, the number of bits needed is still relatively high. A method that allowed even lower bit depth displays to appear similar to displays with higher levels, with noise that is less visible would be useful.